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ABSTRACT 



This paper provides an overview of common sampling methods 
(both the good and the bad) likely to be used in community college 
self-evaluations and presents the results from several simulated trials. The 
report begins by reviewing various survey techniques, discussing the negative 
and positive aspects of each method. The increased accuracy and 
cost-effective nature of probability, as opposed to nonprobability, sampling 
is discussed, as are the different types of probability sampling- simple 
random sampling, stratified random sampling, systematic sampling, and cluster 
sampling. The types of nonprobability samples, including quota sampling and 
convenience sampling, are also described. The paper then reviews the methods 
and subsequent results of the simulation sampling. It concludes that, though 
not the same as collecting fresh data, simulations offer valuable guidance 
and insight to selecting sample populations. Researchers also found that 
random sampling is preferable to in-class surveys, and questionnaires should 
come as close to probability samples as possible. Appendices include data 
tables. (YKH) 



******************************************************************************** 

* Reproductions supplied by EDRS are the best that can be made * 

* from the original document . * 

******************************************************************************** 




T6 °m &S 



Survey Sampling of Community College Students: 
For Better or For Worse 

by 



ERIC 



Richard A. Rasor 
American River College 

& 

James E. Barr 
American River College 

Presented at the Annual Conference of: 
The Research and Planning Group 
for 

California Community Colleges 
Granlibakken - Lake Tahoe 
April 1998 




Pi : education 



originating it. 



PERMISSION TO REPRODUCE AND 
DISSEMINATE THIS MATERIAL HAS 

bee n granted by 



□ Minor changes have been made to 
improve reproduction quality. 



R. A. Rasor 



Points of view or opinions stated in this 
document do not necessarily represent 
official OERI position or policy. 



TO THE EDUCATIONAL RESOURCES 
INFORMATION CENTER (ERIC) 



1 



BEST COPY AVAILABLE 



Abstract 



California’s public community colleges are being held responsible for institutional self-evaluation. 
Such evaluations may be in preparation for accreditation, instructional and service program reviews, or 
simply to gather information about what students think about the various aspects of campus life. The most 
common method of gathering such data is the survey, that is, a questionnaire which usually contains 
several items that elicit some form of student response from an array of several choices that may include 
scaled dimensions of agreement, favorableness, or frequency of occurrence. Sometimes such surveys are 
conducted via telephone or mail while others are administered to students during class time. Researchers 
are often reluctant to use the mail because of disappointing return rates. The telephone would be a logical 
choice, but students are often difficult to reach by phone given their varied hours of school, work, and 
social activities and the survey instrument may be too long for telephone use. Hence, the practice of 
giving a survey in the classroom has endured in spite of frequent objections by some faculty over having 
to give up precious instructional time for the interests of administration or research. This paper does not 
discuss the relative merits of questionnaire construction. Rather, it addresses the practical problem of how 
to reach students. Giving a questionnaire to the entire population of current students is likely to be cost 
prohibitive. Then, how should one sample students given the likelihood that many faculty are not eager to 
relinquish class time, and selecting only certain classes or instructors may yield a student sample that is 
highly unrepresentative? This paper gives an overview of common sampling methods (both the good and 
the bad) likely to be found in community colleges and presents the results from several simulated trials 
illustrating various sampling techniques. They are simulations because no one actually contacted a 
student. It was done by drawing various types of samples from American River College's population of 
students using SPSS 8.0. Detailed demographics and student goals for the population were compared to 
those for each sample with error rates noted. Sampling techniques which resulted in being highly 
representative of the population were stratified random sampling, simple random sampling, and 
systematic sampling (e.g., every 49th name). The sample size in each of these was 400 out of 19,945 
students. Sampling techniques which were not representative of the population and yet were based upon 
far larger sample sizes, were cluster sampling (e.g., class sections or instructors), convenience sampling 
(e.g., multiple days and times; 9:00 AM classes; or introductory psychology classes). It would seem that 
some form of probability sampling (stratified, simple, or systematic) is preferable to the more convenient 
forms of giving a survey instrument in certain classes. The classes contain "captive students" which make 
the researcher's life a lot easier, but the results from such a convenient sample may evolve into 
considerable misinformation about what the college is all about or what directions it should take. It is 
recommended that survey questionnaires be relatively short and administered in such a way that comes 
closest to a probability sample. 
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Survey Sampling of Community College Students: For Better or for Worse 



It is commonplace to periodically collect information from college students regarding their opinions, 
attitudes, beliefs, and practices toward various aspects of campus life. The most common method of 
gathering such data is the survey, that is, a questionnaire which usually contains several items that elicit 
some form of student response - usually by darkening a "bubble" in an array of several choices that may 
include dimensions of agreement, favorableness, or frequency of occurrence. Sometimes such surveys are 
conducted via telephone or mail, while others are administered to students during class time. Researchers 
are often reluctant to use mail as a delivery mechanism because of disappointing return rates. The 
telephone would be a logical choice (as it is with national polling), but students are often difficult to reach 
by phone given their varied hours of school, work, and social activities . Furthermore, any telephone 
survey is best done with only short questionnaires and administrated by trained interviewers - conditions 
that community colleges seem to have difficulty in meeting. Hence, the practice of giving a survey in the 
classroom has endured in spite of the sometimes objection by faculty over having to give up precious 
instructional time for "the administration’s interests" or "research time." Yet surveys will not disappear 
any time soon because there are increasing demands upon community colleges to produce evidence about 
what students think about certain practices - be it evaluation of faculty, courses, programs, or services. 
Institutional accountability measures that are required by both internal and external constituencies usually 
contain measures of what students think and the vehicle for collecting that data is often the survey 
instrument, i.e., the questionnaire. 

The purpose of this paper is not to discuss the relative merits of different forms of questionnaires or 
response menus. Rather, with increasing demands to elicit the opinions of students, the researcher (or any 
person given such a responsibility) must tackle a practical problem, how to reach the students? Giving a 
questionnaire to the entire population of current students is likely to be highly impractical and cost 
prohibitive. Then, how should one sample students given the fact that faculty are not eager to relinquish 
class time and selecting only certain classes or instructors may yield a student sample that is highly 
unrepresentative? 

This paper gives an overview of common sampling methods (both the good and the bad) likely to be 
found in community colleges and presents the results from several simulated trials illustrating various 
sampling techniques. These are called simulations because no one actually contacted a single student. It 
was all done by artful programming and using our student performance database to compare certain 
demographic features of the population compared with the same features of several trial samples. With 
each, the demographic results that were based upon a sample were compared with the actual results based 
upon the population with the degree of error noted. In addition to the usual demographic profile, we 
examined the educational goals of students - both for the population and for each trial sample. In a way, 
this procedure is largely an academic exercise in survey sampling, because in the "real world," one must 
make contact with students who may opt to either participate or not. Still, in spite of not having to 
actually collect fresh data, something might be learned from the amount of sampling error found in these 
simulations. First, we want to review the various types of samples which can be dichotomized into either 
probability or nonprobability samples. Probability samples are those in which there is some way to form 
an estimate of the likelihood that members of a population will be chosen for inclusion within a sample. 
Excellent sources of information about probability sampling may be found in the reference section of this 
paper. In nonprobability sampling , there is no way to estimate this and certainly no high degree of 
assurance that the chosen sample will be close to being representative of the population. 
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Types of Probability Samples 



Simple random sampling : In this type of sampling, every member in the population has an equal and 
independent chance of being selected for the sample. This is much like the common method of drawing 
names out of a hat provided that it is done in an impartial way. The community college researcher must 
have computer access to the entire student population, that is, know how many students there are and how 
to set up a population database. To do this type of sampling, all students in the population are assigned an 
unduplicated number such as social security number. A table of random numbers may then be used to 
generate the numbers to be selected that will correspond to student identification numbers in the 
population. Most statistical software programs (and even some scientific calculators) provide random 
numbers or have a random selection feature. The result should be a student sample that is reasonably 
representative of the student population in terms of their characteristics. 

Stratified random sampling : In simple random sampling, one hopes that the impartial way of selecting 
the sample will also result in a representative sample. To add insurance toward being representative, a 
researcher may use stratified random selection. First, population subgroups are identified that are based 
upon a particular characteristic or strata, e.g., male-female; ages <18 -20, 21-29, 30+years. Each of these 
subgroups are considered as a separate population and random selection is made within each of these. 

For example, assume that a student population at a community college is 40% male, and of these, 10% 
are <18-20 years of age, 25% are 21-29, and 5% are 30+ years of age which sums to the 40%. Also 
assume that the total sample size is to be 400. The researcher now must identify all members of this 
population subgroup who are male and from <18 to 20 years of age. Next, the researcher randomly 
selects 10% or 40 individuals who meet this criterion (.10 x 400). Continuing, 25% or 100 individuals 
(.25 x 400) are randomly selected from the next population subgroup. Finally, 5% or 20 students (.05 x 
400) are randomly selected from that particular population subgroup. Thus the sample reflects not only 
the proportion of males, but the same age proportions as well. In this way, the chosen sample will be 
representative of the population. The major drawback to this approach is that the researcher must know 
all the characteristics or strata of the population in advance of random selecting. Often this is not the case 
and, if not, simple random sampling is a desirable substitute. 

Systematic sampling : This is the label for the type of sample where every n^ 1 member of the 
population is chosen until the desired sample size is reached. For example, assume that the student 
population is 20,000 and listed in alpha order. Also assumed the desired sample size is 400. Then 
dividing 20,000 by 400 equals 50. By randomly starting near the top of the list, every 50th name is 
selected. The "50" is also called the skip number. One may adjust the skip number to reach an exact 
sample size or slightly over the minimum desired size. While the systematic sampling method is generally 
acceptable, it technically does not fit the definition of a strict random sample because not everyone in the 
population has an equal chance of being selected nor is there complete independence. For example, more 
students with the last name of Smith will be chosen than Taylor because there are more Smiths in the 
population. Furthermore, selecting the relatively rare name of Richardsen may influence selection of the 
next person with the name of Richardson because it was skipped over. 

Cluster sampling : In cluster sampling, clusters or units of individuals are randomly selected for the 
sample rather than individuals themselves. The unit making up a cluster may be certain instructors, 
course sections, or even colleges within a population of colleges. Once all clusters are identified in the 
population, a certain number of them are randomly selected from the population of all such clusters. Then 
all individuals within the selected clusters become part of the final sample. As an example, a researcher 
may identify the names of all faculty at a particular community college. She may then randomly select 
10% of the faculty and use all of their students for the sample. The researcher must also eliminate 
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duplicate student names (e.g., when one student is enrolled with more than one instructor chosen as a 
cluster). Suppose a researcher wanted to sample community college students in California. In this case, a 
cluster could be individual colleges. Within the chosen colleges, the researcher may randomly pick 
departments, or even courses in order to reach the students. Such procedures would be called multi-stage 
cluster sampling. 

Types of Nonprobability Samples 

Quota sampling : A quota sample is similar to a stratified random sample with one major difference; 
within each strata, members are selected in a manner that may not be random or representative of the 
population. What matters is to get the necessary proportion (or percent) in the sample that fits the 
research needs. For example, assume one wanted a sample that was proportional to the population and 
that males represented 50% of that population of 10,000 students. Then a sample of 400 students must 
contain 200 males which would also be 50%. It may not be so important how the 200 males were selected 
although it would be difficult to justify using the first 200 males sitting in the college cafeteria. 

Obviously, that would not be a desired sampling technique. However, there are also times when a 
researcher might want to contrast the responses of certain groups. For example, one could randomly 
select 300 whites, 300 Asians, 300 African Americans, 300 Hispanics, 300 Native Americans, and 
another 300 students representing all other ethnic categories. Obviously, this would not be a 
representative sample either, but it would enable the researcher to compare equal sized groups which may 
suit the research purpose. 

Convenience sampling : Any sample that is based upon factors in which convenience plays a major 
role is, by definition, a convenience sample. For example, selecting only students in one particular class 
or students in course sections that meet only on particular days and hours constitute convenience samples. 
Using only people who actively volunteer for a study may also be considered a convenience sample. Such 
samples may be relatively easy to procure but are not likely to be highly representative of the population. 

Other types of nonprobability sampling : Sometimes a researcher will want to target a specific group 
for a survey. An example would be students using the computer laboratory. Such a specific sample is 
sometimes referred to as a purposive sample. A special case of purposive sampling, called snowballing, 
refers to a procedure where a respondent is asked not only to take the survey but deliver it to other people 
who fit the research needs. For example, one of us (Rasor) once asked students in a class to give a survey 
to any divorced male they knew including themselves, if appropriate. In a similar manner, students may 
be asked to enlarge a sample size by asking friends to participate. Admittedly, the results from such a 
"shot gun" survey delivery system cannot be assumed to be representative of a particular population. The 
researcher is obliged to stick with describing the sample and not generalizing to a greater mass. 

How Large a Sample for a College Survey? 

The glib answer is "the more the merrier." But this needs to be qualified. It is true that if the sampling 
is done well, the larger the sample size, the more reliable the results should be. However, there is a point 
at which sufficient accuracy is attained and a further increase in sample size adds little to the accuracy but 
adds substantially to administrative costs. Probability sampling that is done well is cost efficient in that 
the researcher does not need an enormous sample size. Even a national survey of the general public 
requires about 1,500 individuals to be within acceptable error limits. 

In Table 1 are the minimum sample sizes relative to the sizes of a population that are necessary to 
achieve a specified degree of error tolerance. For example, if the population enrollment at a college is 
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20,000 one would need to randomly sample at least 377 students. This count would enable the researcher 
to claim that the results should have no greater than a 5% margin of error and be 95% confident about 
that claim. If one wanted no greater than a 3% margin of error with 95% confidence, the necessary 
minimum sample size would have to be at least 1,033. Note that there is no way of determining the 
necessary sample size for assuring accuracy when using nonprobability sampling. Even a convenience 
sample that is 50% of the population size may dramatically exceed the acceptable margin of error. 
Finally, the sample sizes in Table 1 indicate ending valid counts - not how many questionnaires to 
duplicate or how many phone calls to make. 

Table 1. Selecting the Size of a Random Sample (For Use With a Survey. 

The values listed below indicate the necessary randomly drawn sample size, with a given margin of 
error, relative to the size of the population. The results derived from the sample should be within J- 
5%, + 3%, or+ 1% of the population percentage with 95% degree of confidence. 



Popultion 

Size 


5 % Error 
Sample Size 


3 % Error 
Sample Size 


1 % Error 
Sample Size 










50 


44 


48 


50 


100 


79 


92 


99 


200 


132 


169 


196 


500 


217 


343 


476 


1,000 


278 


521 


907 


2,000 


322 


705 


1,661 


5,000 


357 


894 


3,311 


10,000 


370 


982 


4,950 


20,000 


377 


1,033 


6,578 


100,000 


383 


1,077 


8,926 


1,000,000 


384 


1,088 


9,706 


100,000,000 


384 


1,089 


9,800 



Source: Mitchell, M. & Jolley, J. (1988). Research Design Explained . New York: Holt, Rinehart and 
Winston, Inc. 



Methods 

Participants . All students who were officially enrolled during the fall 1997 semester constituted the 
population. There were a total of 19,945 students. To simplify the extended classification of demographic 
variables, the following categories were established: 

Gender : male or female; 

Age : <18-20; 21-29; 30+ 

Ethnicity : White; African American; Asian + Filipino + Pacific Islander; Hispanic + 

American Indian (Native American); Other. 

Educational goal Transfer; AA/AS degree only; Certificate; Other. 
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In Table 2 is the complete listing of population demographic frequencies and percentages along with 
the stated educational goals for this particular population at American River College. Note that 52.2% of 
the fall 1 997 student population indicated a goal of transfer. 

Table 2. Demography and Goals for Population of Students in Fall 1997 (N= 19,945). 





Gender 


Male 


Female 


Count 


Table % 


Count 


Table % 


Age 


<18-20 


Ethnicity 


White 


1394 


6.99% 


1852 


9.29% 








Afr. Amer. 


163 


.82% 


195 


.98% 








Asian, Fil. t PI. 


334 


1 .67% 


286 


1.43% 








Hispanic, Amer. Ind. 


261 


1.31% 


348 


1.74% 








Other 


87 


.44% 


114 


.57% 




21-29 


Ethnicity 


White 


1981 


9.93% 


2295 


11.51% 








Afr. Amer. 


206 


1 .03% 


314 


1.57% 








Asian, Fil., PI. 


413 


2.07% 


381 


1.91% 








Hispanic, Amer. Ind. 


433 


2.17% 


426 


2.14% 








Other 


131 


.66% 


149 


.75% 




30+ 


Ethnicity 


White 


2088 


10.47% 


3556 


17.83% 






r 


Afr. Amer. 


256 


1.28% 


428 


2.15% 








Asian, Fil., PI. 


302 


1.51% 


391 


1.96% 








Hispanic, Amer. Ind. 


340 


1.70% 


484 


2.43% 








Other 


136 


.68% 


201 


1.01% 



Goal 





Frequency 


Percent 


Valid 

Percent 


Cumulative 

Percent 


Valid 


Transfer 


10413 


52.2 


52.2 


52.2 


AA Degree 


1849 


9.3 


9.3 


61.5 


Certificate 


1122 


5.6 


5.6 


67.1 


All Others 


6561 


32.9 


32.9 


100.0 


Total 


19945 


100.0 


100.0 





Procedures . SPSS 8.0 is our statistical computer package. With each type of probability sample, we 
tried to select a sample size that would be realistically used (e.g., n = 400) and followed strict guidelines 
as called for by the nature of the sample. We also conceptualized the dependent variable or main outcome 
statistic to be the educational goal. Table 2 shown above represents a 2 x 3 x 5 cell division (gender x age 
x ethnicity), or 30 categories. To draw a stratified random sample of size 400 required setting up 30 
subpopulations and randomly drawing within each subpopulation the proportional number. As an 
example, consider white males who fall into the age range of <18 to 20. There were 1,394 such 
individuals in the population which is a proportion of .0699. To properly reflect the population strata, the 
sample proportion must be the same. Then multiplying .0699 times 400 = 28 individuals who are 
randomly selected from the subpopulation of 1,394. This procedure is continued through all strata, i.e., 
for all 30 categories. The percentages within the goal categories of the sample were then compared to 
those of the population. 
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With simple random sampling and with our restrictions on the total sample size, we let the computer 
select the final sample because there is such a program within SPSS. We then examined the percentage 
differences between the population and the random sample demographic variables as well as the 
educational goals. With systematic sampling, we selected every 49th name from an alphabetized list 
which yielded a sample size of 400. We experimented a bit with a random cluster sample of class sections 
in order to reach close to 400 individuals. From that point on, we were guided by selecting various 
nonprobability samples which we have seen before and can best be described as convenience samples, 
e.g., all sections of introductory psychology. After each sampling, we did as before - examined the 
percentage differences between the population and the sample demographic variables as well as the 
educational goals. Standard deviations were computed based upon the distribution of differences between 
percentages. These can be interpreted as an overall index of sampling error. Furthermore, we computed 
chi square values in application of good fit between the sample cell frequencies and those frequencies 
expected by the population proportions. What follows are examples. 

A sample standard deviation is based upon the distribution of differences between sample percents and 
population percents. 

Example: Sample size = 386 



Student Goal: 


Sample% 


Population% 


Difference% 


Transfer 


58.8 


52.2 


6.6 


AA degree 


13.5 


9.3 


4.2 


Certificate 


3.4 


5.6 


-2.2 


All others 


24.3 


32.9 


-8.6 



Standard deviation = 5.9 



The chi square test of good fit compares the obtained frequencies in all cells of the sample with those 
expected which are proportional to the population. 

Example: Sample size = 386 



Student Goal 


Population (p) 


Expected (fe) 


Obtained (fo) 


(fo-fe) 2 

fe 


Transfer 


.522 


201.49 


227 


3.23 


AA degree 


.093 


35.90 


52 


7.22 


Certificate 


.056 


21.62 


13 


3.44 


All others 


.329 


126.99 


94 


8.57 




1.000 


386.00 


386 


22.46 



Chi square tabled .05 value for 3 degrees of freedom =7.815 

Results: Chi square (df= 3, n = 386) = 22.46, p < .001. Because the obtained chi square exceeds the 
tabled value of 7.815, the results are statistically significant. There is not good fit between the sample 
frequencies and those expected from the proportions in the population. In other words, the sample is 
biased to a degree that is likely to be unacceptable. 
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Results of Simulation Sampling 



Rather than display the complete outcomes of every sample as we did with the population (Table 2), 
only summarized results are presented in Table 3. 

Table 3. Summary Statistics Including Chi Square Test of Good Fit for Various Types of Samples. 



Sample Type 


Student 

Sample 

Size 


Demographics 


Student Goa 




Good 

Fit? 


Error Range 

% 


SD 


Chi 

Square 


Error Range 

% 


SD 


Chi 

Square 




Convenience: 
9:00 AM, MWF 


2,077 


-10.8 to 13.4 


3.96 


1,317.70 

(p<.001) 


-17.7 to 24.5 


15.31 


516.05 

(p<.001) 


NO 




Convenience: 
Psych 1 


737 


-10.5 to 19.9 


4.66 


636.51 

(p<.001) 


-14.5 to 22.2 


13.51 


146.05 

(p<.001) 


NO 




Convenience: 
Multiple 
Days & Times 


2,799 


-6.8 to 10.3 


2.82 


945.52 

(p<.001) 


-13.5 to 17.9 


11.33 


376.25 

(p<.001) 


NO 




Cluster: 

Instructors 


10,364 


-3.8 to 3.7 


1.22 


622.49 

(p<.001) 


-7.1 to 10.4 


6.41 


467.17 

(p<.001) 


NO 




Cluster: 

Sections 


386 


-5.5 to 3.1 


1.46 


56.76 

(p<.001) 


-8.6 to 6.6 


5.90 


22.46 

(pc.OOl) 


NO 




Simple 
Random #1 


400 


-2.5 to 1.9 


0.89 


26.33 

(ns) 


-3.7 to 3.6 


2.58 


2.64 

(ns) 


YES 




Simple 
Random #2 


400 


-4.0 to 1.6 


1.00 


31.66 

(ns) 


-3.2 to 2.3 


2.04 


2.60 

(ns) 


YES 




Simple 
Random #3 


400 


-2.5 to 1.2 


0.79 


21.21 

(ns) 


-2.6 to 1.4 


1.55 


1.22 

(ns) 


YES 




Systematic: 
Every 49th 


400 


-1.3 to 1.3 


0.57 


15.52 

(ns) 


-1.9 to 2.5 


1.59 


3.25 

(ns) 


YES 




Stratified 

Random 


400 


-0.2 to 0.1 


0.07 


0.40 

(ns) 


-0.7 to 0.4 


0.41 


0.09 

(ns) 


YES 



The first (and the worst) was a convenience sample consisting of the total student enrollment in all 
9:00 AM - MWF classes. The resulting sample size is 2,077. The demographic error range is expressed 
as two percents, in this case, -10.8% to 13.4%. This indicates that among the 30 possible demographic 
cells described previously, the two extreme differences between the sample and the population were those 
two values. In one cell, the sample percent was below the population percent by 10.8 points (the 
difference). The highest positive difference where the sample percent exceeded the population percent 
was 13.4. The standard deviation of the 30 differences between percents was 3.96. The chi square of 
good fit for the demographic frequencies was 1,3 17.70 which is highly significant. As for the four 
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categories of educational goals, the error range was -17.7% (the sample percent was lower by this amount 
than the population) to 24.5% (the sample percent was higher by this amount than the population). The 
standard deviation of the distribution of percent differences is 15.31 while the chi square test of good fit 
yielded a value of 5 16.05 which is highly significant. This clearly indicates that the 9:00 AM - MWF 
convenience sample was not a good representation of the population in terms of demography or 
educational goal. Even while these 2,077 students are slightly over 10% of the size of the population, 
there is not good fit between the obtained frequencies in the various cells and those expected using the 
population as a model. 

The next sample consisted of all students enrolled in transfer level introductory psychology sections - 
also a convenience sample but one often read about in psychology journals! Here the sample size is 737. 
Both chi square values indicated that there is not good fit between the obtained frequencies in the various 
cells and those expected using the population as a model. 

The third sample in Table 3 is one similar to those used for such things as campus climate or 
accreditation surveys. We extracted an unduplicated student count for those enrolled in any of the 
following sections: 9:00 AM (MWF); 9:30 AM (TuTh); 1:00 PM (MWF); 1:30 (TuTh); and 7:00 PM 
(W). This convenience sample yielded 2,799 students which represented 14% of the size of the 
population. Once again, the sample is quite biased in that the chi square values are highly significant. 

The fourth sample was somewhat of an experiment on cluster sampling. We randomly selected 25% 
of our full-time faculty and 25% of our adjunct faculty. Then the students in all their classes were 
selected for inclusion within the sample. Of course, we also ran an unduplicated headcount. To our 
surprise, we ended up with 10,364 students - 52% of the population size! While it is doubtful that anyone 
would want such a large sample because of the probable costs involved, we decided to run the analysis 
anyway. Both chi squares were statistically significant, indicating that the sample does not reflect the 
characteristics of the population. There is an important point to be made here. Even a 52% size sample 
may not be representative if the units to be in the sample are not directly selected in a random way. 

The fifth sample in Table 3 is a randomly drawn cluster sample of course sections. We tried to keep 
the sample close to our recommended 400 and did with 386 students. While this sample has decidedly 
lower chi square values than the previous samples, they are still statistically significant, meaning that this 
sample was not representative. 

Next are the results of three simple random samples of size 400 each. Notice that the errors are 
relatively small and that none of the chi square values are statistically significant. This indicates that all 
three random samples are acceptable within a tolerated range of error. 

Next to the last is a systematic sample using a skip number of 49, that is, every 49th name from the 
population alpha list was selected to yield a sample size of 400. We had not expected this sample to be 
quite so representative, but it was. In fact, we ran another systematic sample and that was also 
representative. Neither chi square values are statistically significant. 

The last sample in Table 3 is stratified random. The procedure was outlined earlier in this 
paper. The sampling was based only upon demographic strata so we did not expect any meaningful 
differences between the sample and population with respect to these variables. However, we allowed the 
outcome of educational goal to vary, yet it fell right in with the values of the population. In other words, 
that chi square is almost zero indicating excellent fit between the obtained frequencies in the various cells 
of the sample and those expected using the population as a model. 
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Data in Table 4 reflect some of the error inherent in the various sampling strategies. Here we examine 
only the percent of students indicating transfer as their educational goal. In the population, this value is 
52.2%. With a stratified sample of size 400, that value was 5 1.5%, an error difference of only 0.7%. With 
other sampling techniques, the differences are larger although some may still be acceptable. 

Table 4. Percent of Students Indicating a Goal of Transfer to a University 



Population 


N 


% 


Error 










Fall 1997 


19,945 


52.2 





Sample Type 


N 


% 


Error 










Stratified Random 


400 


51.5 


-0.7 










Systematic 


400 


49.8 


-2.4 










Simple Random #3 


400 


54.8 


+2.6 










Simple Random #2 


400 


54.5 


+2.3 










Simple Random #1 


400 


48.5 


-3.7 










Cluster (Sections) 


386 


58.8 


+6.6 










Cluster (Instructors) 


10,364 


62.6 


+ 10.4 










Convenience: Multiple Days/Times 


2,799 


70.1 


+ 17.9 










Convenience: Psych 1 


737 


74.4 


+22.2 










Convenience: 9:00 AM MWF 


2,077 


76.7 


+24.5 



Detailed outcomes from each sampling technique are found in Tables 5 through 14 in the Appendix. 

Conclusions 

We have indicated that running simulated samples is not the same as collecting fresh data. It is more 
like rolling dice in gambling but not having to place a real bet. Yet we hope that these simulations offer 
some guidance in how to go about selecting a sample if population representation is desired. It is obvious 
to us that some form of random sampling (stratified, simple, or systematic) is preferable to the more 
convenient forms of giving a survey instrument in certain classes. The classes contain "captive students" 
which make the researcher's life a lot easier, but the results from such a convenient sample may evolve 
into considerable misinformation about what the college is all about or what directions it should take. 
Finally, we recommend that survey questionnaires be relatively short and administered in such a way that 
comes closest to a probability sample. 
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Appendix 



Table 5. Demography and Goals of Students Based Upon Convenience Sample (All 9:00 AM, MWF, 
N=2007). 





Gender 


Male 


Female 


Count 


Table % 


Count 


Table % 


Age 


<18-20 


Ethnicity 


White 


335 


16.13% 


471 


22.68% 








Afr. Amer. 


42 


2.02% 


35 


1.69% 








Asian, Fil., PI. 


62 


2.99% 


72 


3.47% 








Hispanic, Amer. Ind. 


62 


2.99% 


81 


3.90% 








Other 


26 


1 .25% 


27 


1.30% 




21-29 


Ethnicity 


White 


178 


8.57% 


179 


8.62% 








Afr. Amer. 


19 


.91% 


27 


1 .30% 








Asian, Fil., PI. 


43 


2.07% 


31 


1 .49% 








Hispanic, Amer. Ind. 


33 


1.59% 


27 


1.30% 








Other 


13 


.63% 


7 


.34% 




30+ 


Ethnicity 


White 


65 


3.13% 


146 


7.03% 








Afr. Amer. 


11 


.53% 


22 


1 .06% 








Asian, Fil., PI. 


11 


.53% 


14 


.67% 








Hispanic, Amer. Ind. 


7 


.34% 


22 


1.06% 








Other 


4 


.19% 


5 


.24% 



Goal 





Frequency 


Percent 


Valid 

Percent 


Cumulative 

Percent 


Valid 


Transfer 


1593 


76.7 


76.7 


76.7 


AA Degree 


138 


6.6 


6.6 


83.3 


Certificate 


30 


1.4 


1.4 


84.8 


All Others 


316 


15.2 


15.2 


100.0 


Total 


2077 


100.0 


100.0 
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Table 6. Demography and Goals of Students Based Upon Convenience Sample (All Introductory 
Psychology, N=737). 





Gender 


Male 


Female 


Count 


Table % 


Count 


Table % 


Age 


<18-20 


Ethnicity 


White 


94 


12.75% 


215 


29.17% 








Afr. Amer. 


7 


.95% 


11 


1 .49% 








Asian, FiL, PI. 


26 


3.53% 


30 


4.07% 








Hispanic, Amer. Ind. 


24 


3.26% 


35 


4.75% 








Other 


7 


.95% 


11 


1 .49% 




21-29 


Ethnicity 


White 


56 


7.60% 


65 


8.82% 








Afr. Amer. 


3 


.41% 


8 


1 .09% 








Asian, FiL, PI. 


9 


1 .22% 


7 


.95% 








Hispanic, Amer. Ind. 


6 


.81% 


12 


1 .63% 








Other 


2 


.27% 


4 


.54% 




30+ 


Ethnicity 


White 


19 


2.58% 


54 


7.33% 








Afr. Amer. 


3 


.41% 


5 


.68% 








Asian, FiL, PI. 


4 


.54% 


4 


.54% 








Hispanic, Amer. Ind. 


5 


.68% 


8 


1 .09% 








Other 


3 


.41% 







Goal 





Frequency 


Percent 


Valid 

Percent 


Cumulative 

Percent 


Valid 


Transfer 


548 


74.4 


74.4 


74.4 


AA Degree 


37 


5.0 


5.0 


79.4 


Certificate 


16 


2.2 


2.2 


81.5 


All Others 


136 


18.5 


18.5 


100.0 


Total 


737 


100.0 


100.0 
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Table 7. Demography and Goals of Students Based Upon Convenience Sample (Multiple Days/Times, 
N=2,799). 





Gender 


Male 


Female 


Count 


Table % 


Count 


Table % 


Age 


<18-20 


Ethnicity 


White 


347 


12.40% 


547 


19.54% 








Afr. Amer. 


38 


1.36% 


50 


1 .79% 








Asian, Fil., PI. 


87 


3.11% 


89 


3.18% 








Hispanic, Amer. Ind. 


65 


2.32% 


102 


3.64% 








Other 


24 


.86% 


28 


1 .00% 




21-29 


Ethnicity 


White 


236 


8.43% 


297 


10.61% 








Afr. Amer. 


25 


.89% 


33 


1.18% 








Asian, Fil., PI. 


57 


2.04% 


47 


1 .68% 








Hispanic, Amer. Ind. 


51 


1.82% 


39 


1.39% 








Other 


14 ^ 


.50% 


16 


.57% 




30+ 


Ethnicity 


White 


121 


4.32% 


309 


1 1 .04% 








Afr. Amer. 


18 


.64% 


27 


.96% 








Asian, Fil., PI. 


24 


.86% 


27 


.96% 








Hispanic, Amer. Ind. 


19 


.68% 


40 


1 .43% 








Other 


9 


.32% 


13 


.46% 



Goal 





Frequency 


Percent 


Valid 

Percent 


Cumulative 

Percent 


Valid 


Transfer 


1961 


70.1 


70.1 


70.1 


AA Degree 


222 


7.9 


7.9 


78.0 


Certificate 


74 


2.6 


2.6 


80.6 


All Others 


542 


19.4 


19.4 


100.0 


Total 


2799 


100.0 


100.0 
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Table 8. Demography and Goals of Students Based Upon Cluster Sample of Instructors (N=10,364). 





Gender 


Male 


Female 


Count 


Table % 


Count 


Table % 


Age 


<18-20 


Ethnicity 


White 


999 


9.64% 


1342 


12.95% 








Afr. Amer. 


124 


1.20% 


146 


1.41% 








Asian, Fil., PL 


246 


2.37% 


216 


2.08% 








Hispanic, Amer. Ind. 


188 


1.81% 


243 


2.34% 








Other 


64 


.62% 


76 


.73% 




21-29 


Ethnicity 


White 


993 


9.58% 


1130 


10.90% 








Afr. Amer. 


110 


1.06% 


159 


1.53% 








Asian, FiL, PL 


226 


2.18% 


202 


1.95% 








Hispanic, Amer. Ind. 


216 


2.08% 


228 


2.20% 








Other 


59 


.57% 


62 


.60% 




30+ 


Ethnicity 


White 


820 


7.91% 


1454 


14.03% 








Afr. Amer. 


90 


.87% 


200 


1 .93% 








Asian, Fil., PL 


130 


1.25% 


172 


1 .66% 








Hispanic, Amer. Ind. 


130 


1.25% 


209 


2.02% 








Other 


56 


.54% 


74 


.71% 



Goal 





Frequency 


Percent 


Valid 

Percent 


Cumulative 

Percent 


Valid 


Transfer 


6487 


62.6 


62.6 


62.6 


AA Degree 


831 


8.0 


8.0 


70.6 


Certificate 


369 


3.6 


3.6 


74.2 


All Others 


2677 


25.8 


25.8 


100.0 


Total 


10364 


100.0 


100.0 
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Table 9. Demography and Goals of Students Based Upon Cluster Sample of Sections (N=386). 





Gender 


Male 


Female 


Count 


Table % 


Count 


Table % 


Age 


<18-20 


Ethnicity 


White 


39 


10.10% 


46 


1 1 .92% 








Afr. Amer. 


3 


.78% 


5 


1.30% 








Asian, Fil., PI. 


13 


3.37% 


10 


2.59% 








Hispanic, Amer. Ind. 


9 


2.33% 


10 


2.59% 








Other 


1 


.26% 


3 


.78% 




21-29 


Ethnicity 


White 


17 


4.40% 


42 


10.88% 








Afr. Amer. 






5 


1.30% 








Asian, Fil., PI. 


7 


1.81% 


9 


2.33% 








Hispanic, Amer. Ind. 


10 


2.59% 


13 


3.37% 








Other 


3 


.78% 


3 


.78% 




30+ 


Ethnicity 


White 


36 


9.33% 


68 


17.62% 








Afr. Amer. 


2 


.52% 


11 


2.85% 








Asian, Fi!., PI. 


3 


.78% 


4 


1 .04% 








Hispanic, Amer. Ind. 


3 


.78% 


3 


.78% 








Other 


3 


.78% 


5 


1.30% 



Goal 





Frequency 


Percent 


Valid 

Percent 


Cumulative 

Percent 


Valid 


Transfer 


227 


58.8 


58.8 


58.8 


AA Degree 


52 


13.5 


13.5 


72.3 


Certificate 


13 


3.4 


3.4 


75.6 


All Others 


94 


24.4 


24.4 


100.0 


Total 


386 


100.0 


100.0 
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Table 10. Demography and Goals of Students Based Upon First Simple Random Sample of Students 
(N=400). 





Gender 


Male 


Female 


Count 


Table % 


Count 


Table % 


Age 


<18-20 


Ethnicity 


White 


31 


7.75% 


34 


8.50% 








Afr. Amer. 


3 


.75% 


4 


1 .00% 








Asian, Fil., PI. 


5 


1 .25% 


3 


.75% 








Hispanic, Amer. Ind. 


7 


1 .75% 


6 


1.50% 








Other 


1 


.25% 


1 


.25% 




21-29 


Ethnicity 


White 


34 


8.50% 


41 


10.25% 








Afr. Amer. 


4 


1 .00% 


8 


2.00% 








Asian, Fil., PI. 


9 


2.25% 


6 


1.50% 








Hispanic, Amer. Ind. 


5 


1 .25% 


13 


3.25% 








Other 


2 


.50% 


3 


.75% 




30+ 


Ethnicity 


White 


32 


8.00% 


79 


19.75% 








Afr. Amer. 


3 


.75% 


10 


2.50% 








Asian, Fil., PI. 


10 


2.50% 


14 


3.50% 








Hispanic, Amer. Ind. 


8 n 


2.00% 


14 


3.50% 








Other 


5 


1 .25% 


5 


1 .25% 



Goal 





Frequency 


Percent 


Valid 

Percent 


Cumulative 

Percent 


Valid 


Transfer 


194 


48.5 


48.5 


48.5 


AA Degree 


37 


9.3 


9.3 


57.8 


Certificate 


23 


5.8 


5.8 


63.5 


All Others 


146 


36.5 


36.5 


100.0 


Total 


400 


100.0 


100.0 
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Table 1 1 . Demography and Goals of Students Based Upon Second Simple Random Sample of Students 
(N=400). 





Gender 


Male 


Female 


Count 


Table % 


Count 


Table % 


Age 


<18-20 


Ethnicity 


White 


28 


7.00% 


42 


10.50% 








Afr. Amer. 


1 


.25% 


8 


2.00% 








Asian, Fil., PI. 


4 


1.00% 


6 


1 .50% 








Hispanic, Amer. Ind. 


5 


1.25% 


10 


2.50% 








Other 


1 


.25% 


2 


.50% 




21-29 


Ethnicity 


White 


46 


1 1 .50% 


30 


7.50% 








Afr. Amer. 


5 


1 .25% 


6 


1.50% 








Asian, Fil., PI. 


9 


2.25% 


11 


2.75% 








Hispanic, Amer. Ind. 


12 


3.00% 


5 


1 .25% 








Other 


5 


1 .25% 


5 


1 .25% 




30+ 


Ethnicity 


White 


40 


10.00% 


77 


19.25% 








Afr. Amer. 


3 


.75% 


10 


2.50% 








Asian, Fil., PI. 


4 


1.00% 


6 


1.50% 








Hispanic, Amer. Ind. 


5 


1 .25% 


5 


1.25% 








Other 


2 


.50% 


7 


1 .75% 



Goal 





Frequency 


Percent 


Valid 

Percent 


Cumulative 

Percent 


Valid 


Transfer 


218 


54.5 


54.5 


54.5 


AA Degree 


36 


9.0 


9.0 


63.5 


Certificate 


27 


6.8 


6.8 


70.3 


All Others 


119 


29.8 


29.8 


100.0 


Total 


400 


100.0 


100.0 
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Table 12. Demography and Goals of Students Based Upon Third Simple Random Sample of Students 
(N=400). 





Gender 


Male 


Female 


Count 


Table % 


Count 


Table % 


Age 


<18-20 


Ethnicity 


White 


38 


9.50% 


39 


9.75% 








Afr. Amer. 


4 


1.00% 


3 


.75% 








Asian, Fil., PI. 


7 


1.75% 


6 


1 .50% 








Hispanic, Amer. Ind. 


9 


2.25% 


6 


1 .50% 








Other 


1 


.25% 


3 


.75% 




21-29 


Ethnicity 


White 


37 


9.25% 


42 


10.50% 








Afr. Amer. 


2 


.50% 


5 


1 .25% 








Asian, Fil., PI. 


7 


1 .75% 


6 


1 .50% 








Hispanic, Amer. Ind. 


10 


2.50% 


7 


1 .75% 








Other 


1 


.25% 


2 


.50% 




30+ 


Ethnicity 


White 


46 


1 1 .50% 


71 


17.75% 








Afr. Amer. 


4 


1 .00% 


14 


3.50% 








Asian, Fil., PI. 


4 


1.00% 1 


8 


2.00% 








Hispanic, Amer. Ind. 


2 


.50% 


10 


2.50% 








Other 


1 


.25% 


5 


1 .25% 



Goal 





Frequency 


Percent 


Valid 

Percent 


Cumulative 

Percent 


Valid 


Transfer 


219 


54.8 


54.8 


54.8 


AA Degree 


33 


8.3 


8.3 


63.0 


Certificate 


22 


5.5 


5.5 


68.5 


All Others 


126 


31.5 


31.5 


100.0 


Total 


400 


100.0 


100.0 






4. J. 



20 



Table 13. Demography and Goals of Students Based Upon Systematic Sample of Students: Every 49th 
name (N=400). 





Gender 


Male 


Female 


Count 


Table % 


Count 


Table % 


Age 


<18-20 


Ethnicity 


White 


33 


8.25% 


33 


8.25% 








Afr. Amer. 


1 


.25% 


4 


1.00% 








Asian, Fil., PI. 


6 


1 .50% 


4 


1 .00% 








Hispanic, Amer. Ind. 


4 


1.00% 


7 


1.75% 








Other 


2 


.50% 


3 


.75% 




21-29 


Ethnicity 


White 


37 


9.25% 


47 


1 1 .75% 








Afr. Amer. 


3 


.75% 


4 


1 .00% 








Asian, Fil., PI. 


3 


.75% 


8 


2.00% 








Hispanic, Amer. Ind. 


13 


3.25% 


10 


2.50% 








Other 


2 


.50% 


2 


.50% 




30+ 


Ethnicity 


White 


45 


1 1 .25% 


73 


18.25% 








Afr. Amer. 


6 


1 .50% 


9 


2.25% 








Asian, Fil., PI. 


5 


1.25% 


8 


2.00% 








Hispanic, Amer. Ind. 


5 


1.25% 


14 


3.50% 








Other 


4 


1.00% 


5 


1 .25% 



Goal 





Frequency 


Percent 


Valid 

Percent 


Cumulative 

Percent 


Valid 


Transfer 


199 


49.8 


49.8 


49.8 


AA Degree 


40 


10.0 


10.0 


59.8 


Certificate 


30 


7.5 


7.5 


67.3 


All Others 


131 


32.8 


32.8 


100.0 


Total 


400 


100.0 


100.0 
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Table 14. Demography and Goals of Students Based Upon Stratified Random Sample of Students 
(N=400). 





Gender 


Male 


Female 


Count 


Table % 


Count 


Table % 


Age 


<18-20 


Ethnicity 


White 


28 


7.00% 


37 


9.25% 








Afr. Amer. 


3 


.75% 


4 


1.00% 








Asian, Fil., PL 


7 


1 .75% 


6 


1.50% 








Hispanic, Amer. Ind. 


5 


1 .25% 


7 


1 .75% 








Other 


2 


.50% 


2 


.50% 




21-29 


Ethnicity 


White 


40 


10.00% 


46 


1 1 .50% 








Afr. Amer. 


4 


1 .00% 


6 


1 .50% 








Asian, Fil., PI. 


8 


2.00% 


8 


2.00% 








Hispanic, Amer. Ind. 


8 


2.00% 


9 


2.25% 








Other 


3 


.75% 


3 


.75% 




30+ 


Ethnicity 


White 


42 


10.50% 


71 


17.75% 








Afr. Amer. 


5 


1 .25% 


8 


2.00% 








Asian, Fil., PI. 


6 


1 .50% 


8 


2.00% 








Hispanic, Amer. Ind. 


7 


1 .75% 


10 


2.50% 








Other 


3 


.75% 


4 


1.00% 



Goal 





Frequency 


Percent 


Valid 

Percent 


Cumulative 

Percent 


Valid 


Transfer 


206 


51.5 


51.5 


51.5 


AA Degree 


38 


9.5 


9.5 


61.0 


Certificate 


23 


5.8 


5.8 


66.8 


All Others 


133 


33.3 


33.3 


100.0 


Total 


400 


100.0 


100.0 
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